Using RWE in rare disease drug development: effective innovations with historical controls

Rare disease drug development poses unique challenges that can be overcome by using real-world evidence (RWE). Small populations, paediatric patients, and the desire from patients and their caregivers to receive active therapy can make the conduct of randomised trials with placebo control untenable.

As an alternative, regulatory agencies have allowed the use of RWE to estimate a historical control (HC) compared to patients enrolled in an uncontrolled study. Additionally, healthcare payers understand that HCs can provide value by describing potential treatment benefit relative to untreated patients in their populations. This article will explore the advantages of using HCs, as well as several considerations for their effective use, including scientific methodology and regulatory guidance.

Benefits of historical controls

HCs can offer several advantages over concurrently collected control groups. For example, they can be created from multiple sources of disease natural history information. Disease natural history describes the demographic, genetic, environmental and other variables (eg, treatment modalities, concomitant medications) that relate to the disease’s development and outcomes over its course. Natural history information also describes the current standard of care or emergent care, which may alter disease morbidity and mortality.

Potential sources include ongoing prospective studies such as patient registries, retrospective data from medical charts, control populations from previous trials and expert opinion. There are several rare disease resources which identify ongoing registries and patient communities with medical history information that can be useful as an HC population. These organisations include: Orphanet,¹ a rare disease community resource base with information on over 6,000 rare diseases; EURODIS,² a non-profit alliance of 869 rare disease patient organisations from 71 countries; and NORD,³ a patient advocacy organisation representing patient advocacy groups for over 280 different diseases. This variety of possible sources provides many avenues to find the best source of the requisite information for a given study.

Another benefit is that fewer patients are required in clinical trials using HCs, thereby reducing time to trial completion and speeding drug approval. However, the HC group needs to be very similar to the treated group in all respects, including outcome assessment, disease severity, duration of illness, prior treatments, and other aspects of the current standard of care that could affect the measurement and timing of treatment outcomes. To address these limitations, researchers may use selection techniques that consider independent risk factors that may bias the assessment of the key treatment outcomes, in order to match control subjects to the patients enrolled in the active treatment arm.

HC information can also be used to describe the potential benefit of innovations in treatment over time.⁴ For example, the Early Pseudomonal Infection Control randomised trial described the efficacy of different antibiotic regimens for eradication of newly identified Pseudomonas (Pa) in children with cystic fibrosis (CF). Provision of therapy in the trial was based on culture positivity independent of symptoms. An analysis using HCs was conducted to determine if clinical trial results were different from what would have been observed with a historical standard of care driven more heavily by respiratory symptoms. HCs were identified from the Epidemiologic Study of CF from 1995 to 1998, before widespread availability of tobramycin inhalation solution as standard of care treatment. The trial based antimicrobial therapy for newly acquired Pa resulted in a lower rate of Pa recurrence over a 15-month time period, but comparable hospitalisation rates relative to the HC. This analysis provides an example of how RWE (in the form of a patient registry) can provide evidence to describe the generalisability of trial results within a rare disease population.

Scientific methodology

Two different methodologies are used in the application of HC information in clinical development. The first, Bayesian statistical methodology, uses estimates from historical data to generate a prior probability distribution representing expected event rates for the outcome of interest in a trial, which is combined with the observed data from the clinical trial to update the estimate. Known as the posterior distribution, the estimate is essentially a weighted average of the prior and the observed data from the clinical study. The application of Bayesian methodology has been recognised by the US Food and Drug Administration (FDA) as useful in early phase clinical trials involving paediatric populations.⁵ Applications include more realistic estimates of adverse event rates in small populations; drawing statistical strength from adult data to make decisions about device performance in paediatrics; and shortening trials through use of adaptive designs and predictive probability of trial success before all patients finish the trial.

The second methodology, frequentist methodology, involves the direct comparison of patient-level information from historical data to patients in the clinical study. This provides an approach using similar statistical methods to those used in a randomised clinical trial. The lack of randomisation is likely to introduce bias due to differences between the historical and trial population, but several strategies may help to reduce these differences, including restriction of the patient population entering the HC; matching of the HC to increase similarity with patients in the active arm; and statistical methods to adjust for observed differences. Propensity scores,⁶ which are the probability of treatment assignment conditional on observed baseline characteristics, have been used to adjust for multiple patient factors in non-randomised studies. The propensity score allows for the design and analysis of a non-randomised study so that two or more groups of patients with similar characteristics are compared in the assessment of treatment effect.

The decision to take a Bayesian or frequentist approach to using historical information is generally influenced by the availability of subject-level information to compare with the active trial arm and the preference of regulatory agencies in a specific disease area or application. International regulatory agencies have normally favoured direct comparison of patient level data rather than estimated rate approaches when an HC is completely substituted for a concurrent control arm in a Phase III or registration study.

Regulatory and payer guidance

Regulatory agencies in the US and Europe have been exploring the use of disease natural history information in drug development and the FDA recently published draft guidance in which they outlined the different sources and applications of natural history information, including identification of patient populations, trial planning and the use of HC information to establish drug safety and efficacy.⁷ The European Medicines Agency (EMA) has also published their own guidance for the use of historical controls as part of its guidance on the conduct of trials for small populations.⁸

Healthcare payers also use clinical trial evidence to assess the effectiveness of treatments in making reimbursement decisions. These assessments often need to involve comparisons against multiple alternative therapies that would be impractical in the context of a clinical drug development. The use of HCs can provide a comparative population not studied in the clinical trials. Some formal health technology assessment (HTA) guidance on the use of external controls (Ecs) and RWE is available from the UK’s National Institute for Health and Care Excellence (NICE).^9,10

Patel et al.¹¹ have recently conducted an analysis of international HTA submissions from 2011-2019 to determine the acceptability of single arm clinical studies and ECs as part of the drug reimbursement process. The authors identified 433 assessments done in 21 countries that involved the use of single arm trials (five percent of the total). Overall, 52 percent of these submissions contained some type of supplementary dataset that can be designated as HCs. Based on information available, HCs were most often from prior clinical trials (24 percent), then real-world data (RWD; 20 percent), or both (eight percent). Submissions which contained ECs based on RWD had the most positive recommendations (59 percent), followed by submissions with prior trial ECs (49 percent). This suggests the widespread incorporation and acceptability of HCs in drug reimbursement decisions.

Conclusion

RWE from rare diseases is available from a wide variety of sources for use as an HC in rare disease drug development. HCs are used in drug approvals, HTAs and in describing the impact of new therapies in different clinical settings for rare diseases. As a result, the utility of HCs based on RWE should be evaluated for potential applications to increase speed of product approval, reimbursement and clinical practice adoption.

References

1. https://www.orpha.net/consor/cgi-bin/index.php
2. https://www.eurordis.org/
3. https://rarediseases.org/
4. Mayer-Hamblett N, Rosenfeld M, Treggiari MM et al. Standard Care versus Protocol Based Therapy for New Onset Pseudomonas aeruginosa in Cystic Fibrosis. Pediatr Pulmonol. 2013 Oct; 48(10): 943–953. Published online 2013 Jul 2. doi: 1002/ppul.22693
5. Challenges in Pediatric Study Design and Analysis, and Some Potential Solutions. Thompson, LA. FDA presentation. Available at: https://events-support.com/Documents/Peds_Workshop_2012_v3.pdf
6. Austin PC. An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behav Res. 2011 May; 46(3): 399–424. doi: 1080/00273171.2011.568786
7. FDA Rare Diseases: Natural History Studies for Drug Development Guidance for Industry. Available at: https://www.fda.gov/media/122425/download
8. EMA Guideline On Clinical Trials In Small Populations. Available at: https://www.ema.europa.eu/en/documents/scientific-guideline/guideline…
9. Phillippo DM, Ades AE, Dias S, et al. NICE DSU technical support document 18: methods for population-adjusted indirect comparisons in submissions to NICE. Available at: http://nicedsu.org.uk/wp-content/uploads/2017/05/Population-adjustment-TSD-FINAL.pdf
10. NICE decision support unit Observational data TSD. Available at: http://nicedsu.org.uk/technical-support-documents/observational-data-tsd/
11. Patel D, Grimson F, Mihaylova E, et al. Use of external comparators for health technology assessment submissions based on single-arm trials. Value in Health. 2021Jun8;24(8):1118–25.